The present disclosure relates generally to calibrating multiple 3D sensors to each other and, more particularly, to a multiple 3D sensor installation for a passenger conveyance.
Calibration to a world coordinate system for an individual 3D depth sensor, i.e., to any coordinate system different from an inherent sensor-oriented coordinate system, where its intrinsic parameters are already known, e.g., from manufacturer's data, employs at least 4 locations at different, known, (x,y,z) world coordinates. These 4 locations might be of ‘found’ objects that happen to be in the Field of View (FoV). However, for a professional installation, a portable calibration jig might be carefully emplaced at measured world coordinates, and calibration data of this jig is taken by the 3D depth sensor. With data of the world coordinates (xi, yi, zi) and the corresponding 3D depth sensor observed coordinates, (ui, vi, di), i=1, . . . 4, a transform matrix can be solved to achieve calibration. More than 4 points may be measured for a more robust solution using least squares techniques.
One way to achieve tracking of a potential conveyance passenger is to have one depth sensor with an uninterrupted, unoccluded view from, e.g., a kiosk, to a conveyance, e.g., an elevator car. This is rarely possible due to constraints on the physical space as well as limits to the sensor's FoV. Another approach is to have a few sensors, without overlapping FoVs, and to attempt tracking by re-associating detected potential passengers in the disparate FOVs. This may be difficult to do accurately. Further, accurate arrival time estimation may not be possible without continuous tracking since passenger's speed may vary over time. Finally, tracking may be achieved by having continuous spatial coverage from multiple sensors with overlapping FoVs. This latter approach may require that the sensors have a common understanding of their mutual FoVs which may require potentially laborious and error-prone manual calibration.
An individual sensor's calibration requires determining one or more of the 5 intrinsic and 6 extrinsic sensor parameters. As is well-known, the 5 intrinsic parameters are fx and fy (the focal lengths), x0 and y0 (the principle point offsets), and s (the axis skew). This can be thought of as a 2D translation, 2D shear, and 2D scaling. The intrinsic parameters are typically known, e.g., from manufacturer's data, or may be determined by known techniques prior to installation of the sensor. The 6 extrinsic parameters are x, y, z offsets from the origin of a world coordinate system and the pitch, yaw, and roll angles with respect to the coordinate axes.
In any multiple sensor calibration, some of the parameters may be known and only a subset of the total need to be estimated. To estimate k parameters at least k independent measurements must be made in order to have enough data for a well-defined solution. In particular, if two 3D sensors with known intrinsic parameters are independently calibrated to their respective world coordinate systems, then 6 parameters are required in the most general case (3 translations, 3 rotations) to calibrate the sensors to a common coordinate system.
One issue with calibrating multiple 3D sensors with overlapping FoVs is that any observation between two sensors may vary by inaccuracy in their current individual calibration, by differences in time synchronization between the sensors, and by differences in the sensing of an object (particularly when the object has a 3D size such that different aspects are viewed by the different sensors).
One approach is to manually place one or more small, but still detectable, object(s) in the mutual FOV, and, depending on what, if any, a priori mutual calibration information is known, measure locations while the object(s) remain stationary. With a sufficient number of unique measurements a mathematical equation may be solved to determine the mutual calibration. However, this method may be time consuming and manual measurement may introduce errors.